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ABSTRACT 


The  renewal  function  is  a  basic  tool  used  in  many  probabilistic  models  /  *T,C 

I 

and  sequential  analysis.  Based  on  a  random  sample  of  size  n,  a  nonpara- 
metric  estimator  of  the  renewal  function  is  introduced.  Asymptotic  properties 
of  the  estimator  such  as  the  almost  sure  consistency  and  local  asymptotic 
normality  are  developed.  A  discussion  of  an  application  of  the  estimator  is 
also  provided. 


AMS  (MOS)  Subject  Classifications:  Primary  62G05?  secondary  60K05,  62M99. 
Key  Words:  U-statistics ,  reverse  martingales,  warranty  analysis. 
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SIGNIFICANCE  AND  EXPLANATION 


Let  Xj,  Xj,  •••  be  identically  and  independently  distributed  with 
distribution  function  F.  With  =  X1  +  . . .+  X^,  let  F^^(t)  *  P(Sk  <  t) 
be  the  k-fold  convolution  of  F  for  k  >  1 .  The  renewal  function  H  is 
defined  by 

H(t)  =  ^k>1  F<k>(t) 

for  t  >  0. 

The  renewal  function  is  a  basic  tool  used  in  sequential  analysis  and  used 
in  probabilistic  models  arising  in  areas  such  as  reliability  theory,  inventory 
theory,  continuous  sampling  plans  and  warranty  analysis.  Use  of  the  renewal 
function  is  becoming  widespread  as  efficient  computational  techniques,  which 
can  be  applied  when  the  failure  distribution  is  known,  become  available.  It 
is  surprising  that,  especially  for  small  sample  sires,  estimation  of  the  key 
function  based  on  available  data  has  not  been  addressed  directly. 

Based  on  a  random  sample  of  size  n,  a  nonparametric  estimator  of  the 
renewal  function  is  introduced.  Various  statistical  properties  of  the 
estimator,  such  as  consistency  and  asymptotic  normality,  are  developed.  A 
discussion  of  an  application  of  the  estimator  to  warranty  analysis  is  also 
provided. 


The  responsibility  for  the  wording  and  views  expressed  in  this  descriptive 
summary  lies  with  MRC,  and  not  with  the  author  of  this  report. 


NONPARAMETRIC  RENEWAL  FUNCTION  ESTIMATION 
Edward  w.  Frees 

jj_.  Introduction 

Let  Xj,  X2,  ...  be  Identically  and  independently  distributed  random  vari¬ 
ables  with  distribution  function  F.  Assume  that  F  has  positive  mean  p  and 
finite  variance  cr2 .  With  Sk  -  Xt  +  ...  +  Xk,  let  F(k)(t)  -  P(Sk  <  t)  be  the 
k-fold  convolution  of  F  for  k>l .  The  renewal  function  H  is  defined  by 

H(t)  ‘  ^k>i  F<k)(t)  0.0 

for  t>0.  The  renewal  function  can  be  thought  of  as  the  expected  number  of 
renewals  in  [0,t],  where  the  number  of  renewals  in  [0,t]  is  denoted  by  N(t)  and 
defined  by 

N(t)  "  Xfc>l  I(Sk  <  (1,2) 

Here  I(  •)  is  the  indicator  function  of  a  set.  When  the  observations  X^  are  non- 
negative,  an  equivalent  definition  for  the  number  of  renewals  in  [0,t]  is 
N(t)  -  {sup  k:  <  t}.  The  renewal  function  plays  an  important  role  in  many 

probabilistic  models  (cf..  Feller,  1971  and  Karlin  and  Taylor,  1975)  and  sequen¬ 
tial  analysis  (cf.,  Woodroofe,  1982). 

Most  classical  estimators  of  the  renewal  function  are  based  on  the  assump¬ 
tion  of  a  parametric  form  for  F,  typically  an  exponential  or  Gamma  distribution. 
See  Cox  and  Lewis  (1966)  for  an  early  treatment  of  the  statistical  analysis  of 
renewal  processes.  Most  nonparametric  estimators  of  H(t)  are  based  on  a  reali¬ 
zation  of  the  renewal  process  {x^ } j, and  on  theorems  which  yield  simple  approx¬ 
imations  of  H(t)  for  asymptotically  large  values  of  time  t.  For  example,  sup¬ 
pose  that  the  nonnegative  values  X^  are  recorded  and  that  F  has  an  arithmetic 
distribution.  Recall  that  a  distribution  function  is  said  to  be  arithmetic  if 
its  support  1 s  on  { 0 ,  +  d,  +  2d,  . ..}  for  some  constant  d.  Then,  the  result 

Urn  H(t)  -  t/p  *  (a2  +  p  -  p2)/(2a2) 

t  +* 
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(cf..  Feller,  1968,  p.  341)  has  suggested  the  use  of  the  estimator 

H(t)  -  t/y  +  (o  z  +  M  -  y  )/(2 a  (1.3) 

a  «  j  2 

where  y  and  o  are  estimators  of  y  and  a  based  on  the  data  recorded  up  to  time 

A 

t.  See  Yang  (1983)  for  an  application  of  H(t)  to  continuous  sampling  plans. 
Another  well-known  example  Is  a  functional  central  limit  theorem  for  N(t)  given 
by  Billingsley  (1968,  Theorem  17.3).  Here  the  limits  are  for  t  approaching  «». 

In  this  paper,  estimators  of  H(t)  for  a  fixed  time  t  are  based  on  a  random 
sample  of  size  n,  X^ ,X2 , • • • .Xjj.  Estimation  of  the  distribution  function  F  and 
linear  functionals  of  F  are  problems  that  have  been  thoroughly  Investigated  in 
the  literature  (cf.,  Serf ling,  1980,  chapters  2  and  6).  Viewing  H(t)  defined  In 
(1.1)  as  merely  the  infinite  sum  of  convolutions  of  F,  It  seems  natural  to  esti¬ 
mate  H(t)  based  on  a  sum  of  estimators  of  the  convolutions  of  F.  As  one  would 
suspect,  even  though  estimators  of  the  type  in  (1.3)  are  based  on  recorded 
observations,  they  do  not  perform  well  for  small  (relative  to  y)  times  t.  This 
was  pointed  out  by  Frees  (1984).  In  that  study  the  author  Introduced  several 
estimators,  both  parametric  and  nonparametric,  of  H(t)  for  a  fixed  time  t  based 
on  a  random  sample  of  size  n.  One  nonparametric  estimator  performed  particu¬ 
larly  well  In  the  simulation  portion  of  that  study.  A  variation  of  that  estima¬ 
tor  Is  now  defined.  Let  {i j , • • • *  1^ }  be  a  subset  of  size  k  of  {l,2,...,n}  and 
let  Ic  be  the  sum  over  all  (£)  distinct  combinations  of  {l^ , • • *1^} •  Then,  an 
unbiased  estimator  of  F^^(t)  is 

F(k)(t)  -  (")_1  l  I(X  +  ...  +  X  <  t).  (1.4) 

n  k  c  ik 

Let  m  -  ra(n)  be  a  positive  integer  depending  on  n  such  that  m<n  and  m+“>  as  n+"». 
Then,  a  nonparametric  estimator  of  the  renewal  function  Is 


H  (t)  -  Lm,  F(k)(t). 
n  Tt*»l  n 
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(1.5) 


The  advantages  of  Introducing  the  design  parameter  tn  are  discussed  In  55. 

The  estimator  of  F^k^(t),  F^k^(t),  Is  a  U-statistlc  and  thus  It  Is  easy  to 
establish  that  for  each  k>l  and  for  each  t>0  that 

F(k)(t)  +  F(k)(t)  a.s. 
n 

However,  the  almost  sure  (a.s.)  consistency  of  Hn(t)  Is  surprisingly  difficult 
to  establish.  We  do  so  In  52  by  establishing  that  Hn(t)  *8  8  reverse  martingale 
with  respect  to  an  appropriate  sequence  of  sub  o-fields  plus  some  negligible 
terms.  Also  In  that  section  we  prove  a.s.  uniform  consistency,  the  Glivenko- 
Cantelli  property,  when  the  uniformity  Is  restricted  to  bounded  subsets  of  the 
positive  real  line.  A  counter-example  Is  given  which  shows  that  a.s.  consis¬ 
tency  cannot  hold  uniformly  over  all  of  the  positive  real  line.  In  53,  the 
asymptotic  normality,  when  properly  standardized,  of  H^t)  Is  proved  via  the 
projection  technique  popularized  by  Hajek  (cf.,  Serf  ling,  1980,  Chapter  9.2.5). 
To  keep  potential  applications  for  this  estimator  as  broad  as  possible,  we  dis¬ 
tinguish  between  the  usual  renewal  theory  assumptions  of  nonnegative  observa¬ 
tions  and  the  more  general  framework  which  also  permits  negative  observations. 
The  latter  Is  the  situation  usually  encountered  in  sequential  analysis.  In  54, 
we  prove  the  a.s.  consistency  of  an  estimator  of  the  asymptotic  variance.  This 
provides  the  Important  result  of  large  sample  interval  estimates.  We  conclude 
In  55  by  commenting  on  applications  and  by  providing  an  example  In  warranty 
analysis. 

52.  Almost  Sure  Consistency 

Let  a  e  R  and  g  be  a  real  valued  function  defined  on  R+  ■  (0,“)  such  that 
a 

£  |ga(u)|  d(Ik>1  ka  F(k)(u))  <  ».  (2.1) 


In  this  section  we  establish  the  following  result. 


Theorem  2.1 


Suppose  that  (2.1)  holds.  Then, 

11m  ga(u)  d(£k“jka  F^(u))  -  /"  gfl(u)  ka  F^(u))  a.s.  (2.2) 

In  54  and  55,  we  give  applications  where  a*0.  To  provide  motivation  for  Theorem 
2.1,  we  consider  the  following  corollary  for  the  case  a*0. 

Corollary  2.1 

Suppose  g  Is  a  function  defined  on  R+  such  that 
|g(u) I  dH(u)  <  ». 

Then, 

11m  g(u)  dHn(u)  -  g(u)  dH(u)  a.s. 

n+*° 

Corollary  2.1  Indicates  that  the  sequence  of  random  measures  associated 
with  the  sequence  {h,  Hn,  n>l}  possess  a  type  of  ergodlc  property.  The  state¬ 
ment  of  Corollary  2.1  Is  similar  in  flavor  to  the  statement  of  the  key  renewal 
theorem  of  Smith  (see  1958,  (1.3)).  Some  of  the  applications  of  Smith's  key 

renewal  theorem  are  also  present  In  the  estimation  context  of  Corollary  2.1. 

2 

For  example,  since  H(t)  <  00  for  each  t>0  when  p  is  positive  and  a  is  finite,  we 
may  let  g(u)  *  I(u<t)  to  get  the  following 

Corollary  2.2. 

2 

Suppose  F  has  positive  mean  p  and  finite  variance  a  .  Then,  for  each  t>0, 
Hn(t)  ♦  H(t)  a.s. 

Corollary  2.1,  together  with  some  results  on  uniformly  convergent  measures  due 
to  Rao  (1962),  Is  also  used  to  prove 
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Theorem  2.2. 

2 

Suppose  F  has  positive  mean  u  and  finite  variance  a  .  Then,  for  each  t>0, 

sup  |H  (u)  -  H(u)|  +  0  a. s. 
ue[0,t]  n 

Remarks:  Note  that  Theorem  2.2  does  not  require  that  the  support  of  F  be  on  R+ 
and  also  holds  for  both  arithmetic  and  nonarithmetic  distributions.  The  theorem 
is  a  Glivenko-Cantelli  type  result  and  is  important  in  practice.  A  minor  draw¬ 
back  of  the  result  is  that  the  supremum  extends  only  over  bounded  intervals  and 
not  over  all  of  R+.  That  an  extension  of  the  result  to  R+  does  not  hold  in  gen¬ 
eral  is  given  by  the  following 


Example  2.1. 


Let  he  a  random  sample  with  d.f.  F,  p>0.  Define 

Xnn  -  max{Xj , . . . ,Xn }.  Suppose  F  is  such  that  for  some  e>0  and  for  sufficiently 
large  N,  we  have  Xnn  >  p+e  a.s.  for  all  n>N.  Since  by  the  elementary  renewal 
theorem,  limt+#i(  H( t) /t  -  1/p,  we  have  for  all  n>N, 


8UP+  |Hn(t)  -  H(t)  |  >  |H(n  Xnn)  -  Hn(n  Xnn)| 
t  eR 

-  |H(n  Xnn)  -  m|  n(Xnn/p  ”  “  a.s.* 


The  remainder  of  the  section  is  devoted  to  the  proof  of  Theorems  2.1  and 
2.2.  The  technique  is  to  show  that  /Q  g(u)  d(^™^ka  F^j^(u))  is  a  reverse  mar¬ 
tingale  plus  negligible  terms.  Reverse  martingales  are  a  natural  tool  in  this 

(k) 

context  if  we  note  that  F^  '(t)  is  a  U-statistic.  The  idea  of  applying  reverse 
martingales  to  U-statlstics  is  due  to  Berk  (1966).  An  application  of  Doob's 
(reverse)  martingale  convergence  theorem  will  then  establish  Theorem  2.1.  Let 
{Xi„.  Xjh*  ...»  Xnn}  be  the  order  statistics  associated  with  {Xj ,  •  ••»  xn}. 

We  use  Gn  -  o(Xjn,  •••,  xnn,  X^j,  Xn+2,  ...),  n>l,  to  define  the  sequence  of 
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nonincreasing  sub  o-fields  which  are  implicitly  used  in  all  of  the  following 
reverse  martingales.  We  preface  the  proof  of  Theorem  2.1  with  two  lemmas. 

Lemma  2.1 

Let  g  (  )  be  as  defined  in  (2.1).  Then,  for  each  k>l,  /"*  g  (u)  dP^^(u)  is 
a  o  a  n 

a  reverse  martingale. 

Proof:  It  is  easy  to  see  that  g  (u)  dF^(u)  is  G_-measurable.  Now,  note 
-  o  a  n  11 

that  (2.1)  implies  that  g  (u)  +  0  as  u  -*•  «.  Integration  by  parts  yields 

r  g  (u)  dF^k^(u)  -  g  (u)  F^(u)  I"  -  /*Fnk^(u)  dg  («) 

•*0  a  n  a  n  o  o  n  a 

“  ■  C  Fik)(u)  dga(u>-  (2,3) 

An  application  of  Fubini's  Theorem  using  (2.1)  establishes  that  gft(u)  dF^k^(u) 

is  integrable.  Since  (F^k\u),  Gn}  is  a  reverse  martingale  we  have 

E(F^(u)  |G  ,  )  ■  F^?(u)  a. s.  for  each  u>0.  Thus 

n  n+i  n+i 

E<C  *.<«>  •  -  C  "*.<"> 

-  -  !"o  f2i1)(u>  dga(u)  ■  C  *.(u)  dp£l(u) 

Lemma  2.2 

Let  ga(  )  be  as  defined  in  (2.1).  Then, 

™  c.)  -  £  g,(„>  d(Ik;,k*  p<k)(u))  ♦  Ik>nk*  £  *,(«)  gi(sk<u) 

-  d<Jk>ika  p(l')<>*» 

is  a  zero  mean  reverse  martingale. 


Proof : 


The  proof  Is  similar  to  Lemma  2.1  If  we  note  that 


E(I(Sk<u)|Gn) 


F(k)(u)  k<n 

{  n 

I(Sk<u)  k>n 


a.s  .* 


Proof  of  Theorem  2.1: 

To  prove  (2.2),  It  Is  sufficient  to  show 

lim  r  g  (u)  d(  l  ka  F^k\u))  -  /"  g  (u)  d(£  ka  F^(u))  a. 8.  (2.4) 

nx»  08  k»l  n  08 

To  see  this,  by  Fatou’s  Lemma  and  similarly  to  (2.3),  we  have 


11m  |  /*  ga(u)  d(  l  k8  F<k)(u))  -  /“  ga(u)  ddj^k8  F<k)(u))  | 

k-1 

-11m  r  l  k8  F<k)(u)  d((g  (u)f) 

°  k-uH-1  8 

00 

<  11m  l  /"  k8  F^(u)  d(  |ga(u)  |)  -  0. 

k^itH-1  ‘  ** 


To  prove  (2.4),  we  use  Lemma  2.2  and  Doob's  (reverse)  martingale  convergence 

theorem.  Thus,  there  exists  a  random  variable  Z  such  that  11m  RMn  -  Z  a.s. 

n+°» 

and  11m  E|RM  -  z|  -  0.  From  Fatou's  Lemma, 

n-M» 

0  -  11m  ElRM  -  Z|  >  lim  E(RM  -  Z)  >  E  lim  (RM  -  Z)  -  0 
n  n  n 


and  thus  EZ-0.  Further 
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*  11m  £  /"  ka(F^(u)  -  F^k\u))  d(-g  (u)) 

k-1  °  3 

>  l  m  Q  ka  11m  (F^k)(u)  -  F(k)(u))  d(-ga(xi>)  -  0 

k-1 


Since  ZX)  a.s.  and  EZ-O,  we  have  Z-0  a. a.  This  18  sufficient  for  (2.4)  and  the 
proof . * 


Proof  of  Theorem  2.2: 

Let  A  -  {u>:  uKt  and  ui  la  a  discontinuity  point  of  H}.  Since  the  set  of 

(k) 

discontinuity  points  of  Fv  (•)  Is  countable  for  each  k>l,  A  Is  countable  and  we 
can  let  {a^}  be  some  enumeration  of  A.  Define  g(u)  -  I(u  -  a^).  Since 

g(u)  dH(u)  <  H(t)  <  *»,  by  Theorem  2.1  we  have 


I 

1>1 


I  (rk)(a  )  -  F<k)(a  -)) 
k-1  n  1  n  1 


l  (H(a. )  -  H(a,-)) . 
1>1 


Thus,  without  loss  of  generality,  we  may  assume  that  H(u)  is  continuous  for  u<t. 
The  result  Is  now  Immediate  from  Theorem  2.1  and  Theorems  4.2  and  6.1  of  Rao 
(1962).* 


13.  Asymptotic  Normality 


Def lne 


£  (c)  -  Cov(F(s  c)(t-(X.  +  ...  +  X  )),  F(8-c)(t-(X,  +  ...  +  X  ))).  (3.1) 

I  C  1  C 


In  this  section  we  prove  the  following  result 
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Theorem  3. 1 

Suppose  F  Is  such  that,  for  some  0  >  0, 

/  exp  (~9u)  dF(u)  <  1.  (3.2) 


Let  ra  grow  sufficiently  quickly  so  that  log  n  -  o(m)  and  sufficiently  slowly  so 
that  m  -  o(n*/2).  Then,  for  each  t  >  0 


/“n  (Hn(t)  -  H( t) )  >D  N(0 ,  a2) 


where 


I  I  rs  C  (1). 

r**l  8”1 


Remarks:  Note  that  Theorem  3.1  holds  for  F  both  arithmetic  and  nonarithmetic. 
Further,  there  Is  no  requirement  that  X  be  a  nonnegative  random  variable.  The 
-equireraent  (3.2)  Is  needed  to  ensure  that  ^(S^  <  t)  dies  out  sufficiently 
quickly  as  k  +  «.  Since  p  >  0,  a  sufficient  condition  for  (->.2)  Is  that  the 
moment  generating  function  of  X  exists  In  a  neighborhood  of  zero.  However,  such 
strong  moment  conditions  are  not  always  necessary.  Consider  a  random  variable  Z 
whose  distribution  Is  defined  by  the  probability  density  function,  for  some 
6  >  0, 

f ( z)  -  K  z-3-6  I(z  >  1) 

where  K  is  an  appropriate  constant.  Then,  Var(Z)  is  finite,  (3.2)  is  easily 
satisfied,  and  yet  E  =  ®. 


Using  W  ai  the  duration  of  a  warranty,  suppose  we  are  Interested  in  esti¬ 
mating  the  expected  number  of  renewals  (or  replacements)  by  time  W,  H(W),  or 
more  generally,  we  wish  to  estimate  H(t)  for  t<W.  Suppose  we  replace  the  1.1. d. 
observations  X^  In  the  definition  of  Hn(t)  (see  (1.5))  by  X^  =  mln(X^ ,  W) .  A 
little  thought  leads  us  to  the  conclusion  that  all  the  results  of  $2  through  §4 
remain  valid  with  the  provision  that  the  time  t  under  consideration  is  bounded 
by  W.  This  Is  Important  In  commercial  applications  where  typlcalLy  the  manufac¬ 
turer  of  a  product  only  has  knowledge  of  the  time  to  failure  if  it  occurs  before 
expiration  of  the  warranty. 

In  many  situations  the  choice  of  the  design  parameters  m  and  mj  is  dictated 
by  practical  considerations,  as  in  the  example  above.  Theorem  3.1  gives  some 
theoretical  guidelines  for  the  choice  of  m.  However,  the  convolution  F^)(t) 
dies  out  exponentially  (cf.  (3.10))  as  k  approaches  infinity,  and  typically  m 
can  be  small  compared  to  the  sample  size.  A  similar  argument  can  be  made  for 
m^.  This  Is  Important  since  the  amount  of  computations  increases  quickly  as  m 
(or  mj )  Increases. 


n  this  example, 


line 


rall^H  .a  warranty  and  W  Is  the  duration  of  the  warranty.  T 
reasonable  warranty  duration  ts  the  end  of  the  earLy  failure  period  and  thus  we 
yive  a  point  and  Interval  estimate  of  H(20). 

From  Table  1,  the  estimate  Is  bn(20)  =  p(k)(20)  =  .46194.  Frees 

'  !'3H4)  compared  this  estimate  with  other  estimators  or  H(20)  and  found  It  rea¬ 
sonable.  To  calculate  the  estimated  variance  of  this  estimator,  from  Table  2  we 
have 

°n2  ’  “r , s=  1  "  '  •7538,• 

Thus,  an  approximate  9521  interval  estimate  of  H(20)  Is  Hl05(20)  +  2  o  //i"05 
which  Is  roughly  .46  +_  .17  or  (.29,  .61). 


TARLF,  1 


Convolution  estimates  for  failure  of  a  unit 
of  electronic  ground  support  equipment. 


(20) 
105  J 


.35238 


.09048 


.01684 


.00208 


.00016 


0  0 


TABLE  2 


Covariance  estimates  for  failure  of  a  unit  of 
electronic  ground  support  equipment 


rs 


(1) 


r/s 

1 

2 

3 

4 

1 

.22821 

.05860 

.01091 

.00135 

i 

.01733 

.00490 

.00061 

3 

.00355 

.00048 

4 


00037 


GROUND  SUPPORT  EQUIPMENT 
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etrlc  estimation  of  this  key  function  based  on  the  available  data.  The  simula¬ 
tion  study  of  Frees  (1984)  showed  that  an  estimator  similar  to  ^(t)  performed 
well  for  small  (n  <  30)  sample  sizes.  An  example  in  warranty  analysis  Is  given 
below.  The  techniques  of  this  paper  may  also  be  useful  In  sequential  analysis. 
For  example,  an  Important  parameter  In  sequential  estimation  Is  the  mean  of  the 
renewal  function, 

P  ■  u  dH(u) , 

see  Woodroofe  (1982).  Defining  u  dF^^(u),  from  Corollary  2.1  we 

have  that  p  Is  an  a.s.  consistent  estimator  of  p  if  p  <  °°.  Another  example  is 
n 

the  expected  value  of  the  first  passage  time 
t  -  inf  (n  >  1:  S  >  0}. 

From,  for  example,  Woodroofe  (1982,  Corollary  2.4)  we  have 
E(t)  -  exp{^k>1  k  1  P(S^  <  0)}  when  u  >  0.  Thus,  by  Theorem  2.1, 

xn  -  exp  { k  1  F^  (0)}  Is  an  a.s.  consistent  estimator  of  E(x).  We  intend 
to  explore  other  applications  of  nonparametric  renewal  function  estimation  in 
sequential  analysis  In  another  paper. 

To  Illustrate  how  to  calculate  the  estimator,  we  used  observations  of  the 
time  to  failure  of  a  unit  of  electronic  ground  support  equipment  which  were  pre¬ 
viously  used  by  the  author  (Frees,  1984).  The  data  can  also  be  found  in  Juran 
and  Gryna  (1970,  p.  171)  and  Kolb  and  Ross  (1980,  p.  170).  In  Graph  1  an  esti¬ 
mate  of  the  failure  rate  curve  is  given  which  suggests  an  early  failure  rate  of 
about  20  hours.  The  estimate  of  the  failure  rate  was  based  on  Epanevicoch’s 
method.  The  calculations  were  done  on  a  VAX  11/750  owned  and  operated  by  the 
Department  of  Statistics  at  the  University  of  Wisconsin-Madison.  Now,  it  is  not 
unusual  for  the  manufacturer  of  equipment  to  enter  into  an  agreement  to  replace 
the  equipment  for  a  certain  length  of  time,  say,  W.  This  type  of  agreement  Is 


19 


To  prove  (4.4),  we  first  establish  an  analogue  to  Lemma  2.2.  Let 


R„‘  I  "{r8+  I  r3  EfI(Sr  ‘  l)  KSr+.-l  "  Sr-1  '  1)|<3nl 

r+s- 1 <n  r+s-l>n 

'  I  ”  «rs- 

r,  s>l 

It  is  easy  to  see  that  is  Gn-measurable  and  integrable.  Further,  since 


E(Efl(Srtt>  I(Srfa_1  -  Vl  *  t>l<:„HOn»1) 

-  E(I(Sr<t)  KSr+g_1  -  Sr_1  <  t)|Gn+1)  a.s. 


A 

and  for  r+s-1  <  n,  E(I(Sr  <t)  KS^^j  <  t)|Gn>  =  £rg  a.s.,  ^  is  a  zero 

mean  reverse  martingale.  Now,  as  in  the  proof  of  Theorem  2.1,  an  application  of 
Doob's  (reverse)  martingale  convergence  theorem  and  repeated  application  of 
Patou' 8  Lemma  yields 

11m  l  rs  ire  +  l  rs  K(I(Sn  <  t)  -  S^j  <  t)|Gn) 

n«  r+s- 1  <n  r+s-1  >n 


£r,s>l  rs 


Thus,  to  complete  the  proof,  we  note  that 

lim  l  E(I(Sr  <  t)  I(Sr+8_1  ~  Sr-1  <  t)|Gn)  -  0  a. 

n+<»  r+s-l>n 


11m 

n+«  r+s 


a  in  i  a 

£  rs  5  ~I  .  rs  5 

- 1  <n  ”  r-s-‘ 


0  a.s. + 


15.  Concluding  Remarks 

The  renewal  function  arises  in  a  wide  variety  of  applications  of  probabil¬ 
istic  models  such  as  in  reliability  theory.  Inventory  theory  and  continuous  sam¬ 
pling  plans.  In  this  paper  we  have  presented  the  asymptotic  theory  of  nonparam- 
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\  '  Jr.i-l  £r.  '  <Cl  1  <k)U»2. 


(4.3) 


we  have 


Remarks:  For  a  general  F,  by  Theorem  3  of  Baum  and  Katz  (1965),  E  |x[ 3  <  •  la  a 
sufficient  condition  for  (4.2).  Further,  from  (3.10),  it  is  easy  to  see  that 

(3.2)  is  sufficient  for  (4.2).  Thus,  we  have 

Corollary  4.1. 

2 

Assume  that  the  conditions  of  Theorem  3.1  hold  and  let  o  be  defined  as  in 

n 

(4.3) .  Then, 

*rn/on  (Hn(t)  -  H(t) )  +D  N(0, 1) 
and  thus,  for  a  e  (1/2,1), 

lim  P(Hn(t)  -  za/2  <  H(t)  <  Hn(t)  +  za/2  an//~n)  -  1-a 


where  z  is  the  a/2  quantile  of  the  standard  normal  distribution. 


Proof  of  Theorem  4.1: 

From  Theorem  2.1,  with  a-1  and  ga(u)  -  I(u<t),  we  have 


U  k  Fnk>(t>  *  Ik.I  k 


Thus,  we  need  only  show  that 


^r,8-l  rB  ^rs  *  ^r,s>l  rs 


(4.4) 


.v  *■  ;■ 


•  V  V  ’/  •*  V  ’  j  V  v 

-  * u  v  •  i  *  -  «  <• 
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This  result  and  Theorem  3.1  will  immediately  provide  a  confidence  interval  for 
H(t). 

Let  5  -  E(F(r_1)(t-X)  F(s-1)(t-X)).  We  wish  to  estimate 

rs 


2 

o 


L 


,s>l 


rs 


5r.(1> 


-  8>1rs  (F  (F(r_1)(t-X)  r(s""  (t-X>)  -  F(r'(t)  F(s,(t)) 

-  L,.»I  "  {r,  -  <L>1  r  r(r,<'»2- 


To  define  an  estimator  of  £  , 

rs 

not  necessarily  ordered.  Let  ][ 
subsets  of  size  r+s-l  from  {l,2 


let  {ij ,i2 , . . .  ,i  }  be  a  subset  of  {l,2,...,n}, 
denote  the  summation  over  all  n!  permutations  of 
,...,n}.  An  unbiased  estimator  of  is 


(n-r-s+l) !/n!  ^  1(XA 


. . .  +  x1  <  t) 
r 


I(X.  +  X  +  ...  +  X.  <  t). 

1  r+1  r+s-l 


(4.1) 


In  this  section,  we  prove  the  following  result. 

Theorem  4.1 

Assume  that 

Ik>l  k  F(k) (t)  <  ».  (4.2) 

Let  raj  -  mj(n)  be  a  positive  Integer  depending  on  n  such  that  mj<n  and  njj  +  ~  as 
n  +  «.  Then,  with 


To  prove  (3.12),  note  that 


n  (n)_1  (8)  (n_8) 
r  c  r-c 


r!  s!  (n-r) !  (n-s) !  {(r-c)!  (s-c)!  c!  (n-r-8+c)!  ( n— 1 ) ! } 


-1 


A(n,r,s)  (*)  (8)  c!  (n-r-s+1)!  /  (n-r-s+c)! 


Note  that  c!  (n-s-r+l)!  /  (n-r-s+c)!  *  0(n-*), 


raln(r,s) 

£c-l  O  (*)  ■  (  „S).  sup  A(n,r,s)  <  1  +  o(i),  and 


c  c 


5^(0  |  <  F<r>(t)  F<8>(t).  Thus, 


U-l  "  (r>"  &  O  O  W'>  I 


<  0(n_1)  I,*.,  P(r)(t)  [c;2  (')  ([) 


<  0(»-*)  Ir  (E  .-«)«  <”)  -  Otn’1) 


-0X  N  r+s  , r+s - 


by  (3.10)  and  the  fact  that  for  p<l, 


Jr.;.,  P™  <7>  <  -♦ 


84.  Interval  Estimates 

A  local  asymptotic  normality  result  such  as  Theorem  3.1  Is  appealing 

because  It  gives  Information  about  the  rate  of  convergence  of  Hn(t)  to  H(t). 

However,  in  applications  it  is  also  desirable  to  give  interval  estimates  of 

2 

H(t).  In  this  section  we  present  an  estimator  of  o  and  prove  Its  almost  sure 
consistency.  In  the  proof  the  reverse  martingale  technique  of  82  is  utilized. 


Lemma  3.3 


Under  the  assumptions  and  notation  of  Theorem  3.1, 


L,:.l  I”  <;>-*  Ic-I  <=>  <«>  "  W»  *  0 


Proof: 


Sufficient  for  the  proof  is 


and 


ir.j-i  <"  <;>'*  <;>  <”>  -  «>  w»  *  °- 


To  prove  (3.13),  by  Lemma  3.2,  we  have 


n  (”)  1  s  -  rs  -  rs  ((n-s)!  (n-r) !  {(n-1)!  (n-r-t 


■  rs  (A(n,r,s)  -  1). 


Thus,  by  Patou's  Lemma, 


n.  !  Ir_,  (n  (J)-1  .  <™>  -  »)  ersu> 


<  L  rs  I  C  ( 1 )  |  iim  sup  I  A(n,r,s)  -  1 


'.1.  '.-V  '.■v  ”•' ’ '.  *■ '  11  ■  ’  «."  r*  I.’1’  ■  i.'  I.'  rj  •’  »  ■  '> 1  ■  1  »  ■  ■  u  » ■ 
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Proof : 


Recall  the  inequality  (cf.,  Feller,  1968,  p,  54), 

*r2 ^  (n  +  l/2)n+1/2  exp{-(n  +  1/2)  -  1/24/n  +  l/2)“l } 

<  n!  <  /~ii  (n  +  m)"*1'2  expf-(n  +  1/2)}. 


Thu  8 , 


A(n,r,a)  <  •<"  -  r  *  <°  ~  ■  *  !■  +  °0>l, 

(n  -  <n  -  r  -  ■  +  3/2)"”r'*+3/2 
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Define  nj  -  n-1/2  and  n2  “  n-r-s+3/2.  Then, 


A(n,r,s)  <  (1  -  (r-D/nj)”1  (1  +  (r-l)/n2)"2 

(1  +  (r-s)/(n-r+l/2))r  {l  +  o(l)}. 

Thu  8 , 

sup  log  A(n,r,s)  <  sup  (n.  log(l  -  (r-l)/n. ) 
r,s  r,s  1 

+  n2  log(l  +  (r-l)/n2)  k  log(l  +  (r-s)/(n-r+l/2))}  +  o(l) 

■  sgp  -(r-1)  +  o(l)  +  (r-1)  +  o(l)  -  o(l). 

Thus,  lira  sup  A(n,r,8)  <1.  The  inequality  in  the  opposite  direction 
r,8 


is  similar.# 


Thus,  from  (3.2), 


u 


/T  (H(t)  -  H*(t) )  -  fh  Ik>m  F(k)(t) 

» —  Qtf-f  -0XNNm  /,  -0XNX— 1  _ 

<  /  n  e  (E(e  ))  (1-E(e  ))  ♦  0 


by  an  easy  application  of  L’Hospital's  rule  since  log  n  ■  o(m).  Thus,  suffi¬ 
cient  for  the  proof  of  the  lemma  is 

/"n  (Hn(t)  -  H*(t) )  >D  N(0,  a2).  (3.11) 

To  prove  (3.11),  from  (3.5),  define 

Unj  "  n_1  Ek-1  (F(k"1)<t-XJ)  -  F(k)(t)). 


Now  {l7n  j ;  J-l,...,n,  n>l }  is  a  double  array  of  random  variables  that  are  inde¬ 
pendent  within  rows.  Now  E  Unj  -  0  and,  by  (3.6), 

rs  5rs(l).  Because  of  the  strong  moment  conditions, 

it  is  easy  to  check  that  the  usual  uniform  asymptotic  negligibility  and 
Lindeberg  conditions  hold  (cf.,  Serfling,  1980,  51.9.3).* 


Var(  1  U  ,)  -  n  1  l 


Lemma  3.2 

Define  A(n,r,s)  «  (n-r) !  (n-s) !  { ( tt— 1)1  (n-r-s+1)!}  * .  If  r-o(n*^)  and 
s*o(n^2),  then 


lira  sup  A(n,r,s)  ■  1 
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Hence,  from  (1.5)  and  (3.5), 


Cov(H  (t) ,  H(t))  -  L"  Cov(P  (r) (t) ,  s/n  J  P^8'i;(t-X  )) 
n  n  r,a-l  n  J 


?  »<8-l), 


■  ^r,i-l  r,/”  Er.(1)' 


(3.8) 


Thus,  from  (3.6)  -  (3.8), 


n  e(H„(t)  -  Hn(t))2  .  n(v«r(Hn(t))  +  V»r(Hn(t))  -  2  Cov(Hn(t),  i,(t>)l 


K.'-l  1“  0“  \  O  Er.<‘>  -  ™  Er.(1)l- 

’  C*1 


(3.9) 


We  now  present  a  series  of  lemmas  which,  when  taken  together,  provide  a 
proof  of  Theorem  3.1. 

Lemma  3.1 


Under  the  assumptions  and  notation  of  Theorem  3.1, 


(Hn(t)  -  H(t) )  N(0,  <7  ) . 


Proof : 


By  the  Markov  Inequality,  for  9>0,  we  have 


P(k)(t)  -  P(-0  Sfe  >  -0  t)  <  e6t  (E  e_9X)k. 


(3.10) 
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E  Itt  +  X  +  ...  +  X  <  t)  I(X.  +  X.  +  ...  +  X.  <  t) 
al  a2  ar  bl  b2  \ 

-  F(r)(t)  F(s)(t)  -  5rs(c). 

Thus, 

Cov(F(r)(t),  F(fl)(t)) 
n  n 


O'1  O"1  E  L  L  {(I(X«  +  +  Xa  <  t)  -  F^r;(t)] 

r  8  c  c  ar 


[I(X  +  ...  +  Xb  <  t)  -  F(fl)(t)]} 
D1  s 


(jr1  l*  (!)  (i;!)  5  (c) 

r  cmo  c  r-c  rs 


since  the  number  of  distinct  choices  for  two  subsets  of  size  r  and  s,  respec¬ 
tively,  having  exactly  c  elements  In  common  is  (")  (8)  (^_a).  Thus, 

'«•»  -  Ir.T-l  (£c>  fr.M-  <3‘7) 

a  C  r) 

To  calculate  CovCH^t),  H^(t)),  we  first  examine  the  covariance  between  Ffl  (t) 

and  p(®_1 ) (t-x: ) .  Now, 

Cov(F<r)(t),  F(8_1)(t-X,)) 
n  l 

*  O"1  {(JlJ)  Cov(F^r_1  )(t-Xj) ,  F(s_1)(t-X1))} 


"  r/n  5rs(1)* 
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Prom  the  definition  of  F^  '(t)  In  (1.4),  an  easy  calculation  ahoira  that 
E(P^k)(t)|Xj)  -  P^^t-Xj)  +  (n^)  P(k>(t)} 


(k/n)  F(k_1)(t-X1)  +  (1-k/n)  F(k)(t). 


(3.3) 


Define  a  truncated  version  of  H(t),  H  (t)  ■  J  Pv  ^(t).  We  define  the  projec- 

k**l 

tlon  of  H^t)  on  H*(  t)  by 


Hn(t)  -  l  E(Hn(t)|X1)  -  (n-1)  H*(t). 


(3.4) 


From  (3.3),  we  have 


r 


H  (t)  -  H*(t)  -  n-1  I  l  k{F(k-1)(t-X.)  -  F(k)(t)}.  (3.5) 

j-1  k-1  1 

The  Idea  of  projecting  H^t)  onto  the  original  Independent  obaervatlona  la 

A 

due  to  Hoeffding.  Since  Hn(t)  is  Just  the  sum  of  n  independent  random  vari¬ 
ables,  the  usual  theorems  for  double  arrays  of  independent  random  variables  are 

A 

used  to  obtain  a  limiting  asymptotic  distribution  for  H  (t).  We  then  show  that 

n 

A 

the  moments  of  Hn(t)  “  Hn(t)  are  small  in  the  appropriate  sense  to  get  an  iden¬ 
tical  asymptotic  distribution  for  H^t). 

From  (3.1)  and  (3.5),  we  have 


Var(Hn(t))  -  "  1  rs  Cov(F(r_1) (t-X) ,  F(*_1)(t-X)) 


m  ,  rs  £  (1) 

Lr,s-1  rs 


(3.6) 


To  calculate  Var(H_(t)),  we  first  examine  the  covariance  between  F'r;(t)  and 
11  n 

F^(t).  Let  (ap  ,  ...,  arJ  and  {bj,  bj ,  ...,  bg}  be  two  subsets  of 
{l,  2,  ...,  n}  that  have  c  <  mln(r,s)  elements  in  common.  Then, 
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